#currentqueue .net threading

[manishdhoj-blog]

Thread and ConcurrentQueue

ConcurrentQueue is a thread safe queue in .NET. We use queue to decouple tasks and also to process things faster. Recently I had to work on a task where you fetch records from a xml file,parse it and then map it into database tables. First I tried to approach this with regular procedural process. This took way longer than I had expected. In fact I was not able to run the complete process even once. I had to kill the process after 12 hours of baby sitting.

After researching, I landed on this object called ConcurrentQueue. also I decided to use thread to make the processing faster. Concurrentqueue is a thread safe queue. to summarize the steps.

1. Create a structure that contain all the essential objects for processing the task.

2. Create a concurrent queue.

3. Create a Task with the item to be queued.

3. Create an array of  Task. This asynchronously processes the method.

Also one main change I had to do is to move away from the individual read/write and move towards block read/write. I had used the EF save method in the beginning but later changed to call stored procedures passing the DataTable.

Below is the code snippet:

 private void ProcessItemWithQueue(ConcurrentQueueItem myConcurrentQueueItem)

{ _sharedQueue= new ConcurrentQueue<ConcurrentQueueItem>();  var taskToQueue = Task.Run(() =>  {  _sharedQueue.Enqueue(myConcurrentQueueItem);  }); Task[] tasks = new Task[2]; for (int i = 0; i < tasks.Length; i++)            {tasks[i] = new Task(() =>  {      ProcessQueueItems(_sharedQueue, taskToQueue); }); tasks[i].Start(); } taskToQueue.Wait(); Task.WaitAll(tasks); }

 private void ProcessQueueItems(ConcurrentQueue<ConcurrentQueueItem> queue,Task taskToQueue)        {            do            {                while (!queue.IsEmpty)                {                     ConcurrentQueueItem myUnitOfWorkDequed;                    bool gotElement = queue.TryDequeue(out myUnitOfWorkDequed);                    if (gotElement)                    {                        Process(myUnitOfWorkDequed);                    }

             }

           } while (!taskToQueue.IsCompleted);

       }

After making these changes to my code I was able to complete the process in about half an hour. Mission accomplished.